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ABSTRACT 



This paper uses an approximate average percent-correct methodology to compare the ranks that would 
be obtained for PISA 2006 countries if the rankings had been derived from items judged by each country to 
be of highest priority for inclusion. The results reported show a remarkable consistency in the country rank 
orderings across different sets of countries’ preferred items when comparing with the rank reported in the 
PISA 2006 initial report (OECD, 2007). On average, only few countries systemically go up or down in 
their ranking position. As these countries are in a group of moderate performers with very comparable 
outcomes, these shifts in the ranking would probably be associated with minor changes in mean 
performance on the final PISA scale. The analysis suggests that PISA rankings are noticeably stable thanks 
to the large enough pool of test items able to accommodate diverse preferences. The paper shows how 
important it is to base a choice of test items on a properly structured process which allows different experts 
and countries to equally contribute. The evidence presented demonstrates that in PISA, average rank 
positions of countries across different sets of preferred items are apparently stable and experts are not able 
to predict which items can elevate performance of their countries in the final test. 



RESUME 



Le present document repose sur une methodologie fondee sur la moyenne des pourcentages de reponses 
correctes. II vise a determiner le rang que les pays auraient obtenu a revaluation PISA 2006 si le 
classement avait ete realise a partir des items consideres comme prioritaires par chaque pays. Sur les 
differents groupes d’items preferes des pays, les resultats montrent une coherence remarquable avec le 
classement reel qui figure dans le rapport initial PISA 2006 (OCDE, 2007). En moyenne, peu de pays 
gagnent ou perdent des places systematiquement. Etant donne que ces pays font partie d’un groupe de 
niveau moyen avec des resultats tres comparables, les decalages dans le classement s’accompagneraient 
probablement de changements mineurs dans la performance moyenne sur l’echelle finale PISA. L’analyse 
suggere que les classements PISA sont manifestement stables grace a l’existence d’un vivier d’items de 
tests suffisamment important pour autoriser toutes les preferences. Ce document montre qu’il est important 
de fonder le choix des items de tests sur un processus bien structure, ce qui permet aux pays et aux experts 
de contribuer de la meme fatjon. Les observations presentees ici etablissent que pour PISA, le classement 
des pays selon les differents groupes d’items de premier choix est apparemment stable et les experts ne 
peu vent pas predire quels items pourraient gonfler la performance de leur pays dans les tests finals. 
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ANALYSIS OF PISA 2006 PREFERRED ITEMS RANKING USING THE PERCENT-CORRECT 

METHOD 



Introduction 

1 . The PISA international results are based on a scaling of the raw data with an item response model 
(see PISA 2006 Technical Report). This scaling is undertaken for a number of reasons. First, and perhaps 
most importantly it supports the construction of described proficiency scales. Second, it enables the 
summarisation of students’ responses to many items with simple indices. In doing so it ensures that the 
indices reflect the fact that distinct sub-samples of students respond to test booklets that are made of 
different subsets of items. 1 

2. Through the application of the scaling model the PISA 2006 country level scores of each PISA 
domain are derived from all of the items in the PISA 2006 item pool that pertain to that domain. 2 This 
procedure reflects the PISA approach, which was one of building internationally supported assessment 
frameworks and then developing items pools that sample widely from those frameworks in an agreed 
fashion. 

3. This approach does, however, have the potential to hide possibly interesting differences in 
patterns of performance at lower levels of aggregation. The study reported here investigates the impact of 
the selection of science items on the ranking of the countries. The analysis is preliminary because it uses an 
approximate percent-correct methodology. 

4. As part of the PISA 2006 item development process each country was invited to rate each of the 
Field Trial items according to their priority for inclusion in the PISA 2006 Main Study. This data was used 
by the Consortium in the process of selecting Field Trial items for inclusion in the Main Study. The 
priorities for inclusion ratings were provided on a five -point scale (1 = low, through to 5 = high) (see PISA 
2006 Technical Report for details). 

5. Here we use the results of these ratings to develop multiple sets of overall science rankings for 
the PISA 2006 countries - one based on each country’s preferred set of items. The rankings are based upon 
an average percent-correct methodology, which is an approximation to a more complete scaling of the data. 

Average Percent Correct Computation 

6. For each item administered in PISA it is possible to compute the percentage of correct answers. 
This is simply the number of correct answers obtained in a particular country divided by the total number 
of students who responded to the question. 3 We call this statistic the item -p. 



1 The use of scaling models also makes possible a range of statistical checks of the reasonableness of the data. A discussion of this is not however 
relevant here. 

2 With the exception of a small number of national level item deletions described in the PISA 2006 Technical Report. 

3 To be more precise the statistic is the proportion of the total possible score, since the denominator is the sum of the maximum possible scores 
across all students who responded to the item, and the numerator is the sum of the observed scores. This makes the statistics suitable for use 
with items scored out of one or more. Further the summations are weighted with the PISA student weights. 
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7. The item -p statistics can be averaged over any subset of items to obtain an average-p for that 
item subset and countries can be ranked according to that average. This ranking usually provides a close 
approximation to the ranking that would be obtained from an application of the PISA scaling 
methodology. 4 

Priority for Inclusion 

8. Table 1 shows the number of Main Study science items that were rated at the highest level 5 
(very high priority for inclusion) by each country. 

9. The second column presents the highest rated level by country. Some countries didn’t rate any 
item at level 5. For these countries, namely Macao-China, Montenegro, and Slovak Republic, items ranked 
level 4 were considered as the most preferred. The number of items rated at the highest level by country is 
given in the third column. 

10. One hundred and three science items were used in the PISA 2006 Main Study. These items are 
considered here. However, 22 items were not tested in the Field Trial and consequently were not rated by 
countries. These are items taken from previous PISA cycles which were included to allow score linking 
across PISA cycles. Thus, two sets of results are provided. The first based solely on new PISA 2006 items 
rated by countries at the highest level for inclusion; and the second based on the 22 items from previous 
cycles plus countries’ highly preferred items developed for PISA 2006. There are more items in the second 
set making it more stable and better suited for comparisons. The discussion concentrates on the second set 
of results, while data are provided for both. 

11. Table 1 does not list all countries participating in PISA 2006. Some countries did not provide 
these ratings or rated all items at the highest level. 5 Additional statistics for all participating countries with 
comments explaining why some of them were not included in the analysis are presented in the table A 1 in 
the Appendix. 

12. In the analysis considering only new PISA 2006 items, data from countries that rated less than 
five items at the highest level were not used. Spain and Switzerland gave the highest ranks to only two 
items and were not considered, because any scores based on two items are too volatile to be compared. 
However, these two countries were considered in the analysis where items from previous cycles were 
included. 



4 Small differences will occur for three reasons. First, the average percent correct methodology weights all items equally, whereas in the scaling the 
items are weighted according to the number of booklets that they were included in. Second, the average percent correct approach does not address 
the booklet effect that was observed in PISA. Third, the scaling methodology corrects for that fact that percentage values are bounded at zero and 
100, and therefore cannot be regarded as interval. 

5 A number of countries that did not provide ratings did so on the grounds that they deemed all items as appropriate for inclusion. 
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Table 1 . Number of items rated the highest priority for inclusion by each country 



Country 


Items Rated Level 5 


Maximum rated level 


Items rated at the highest 
level in the country (5 or 4) 


Australia 


50 


5 


50 


Austria 


14 


5 


14 


Belgium 


18 


5 


18 


Canada 


28 


5 


28 


Chile 


22 


5 


22 


Chinese Taipei 


50 


5 


50 


Colombia 


58 


5 


58 


Croatia 


63 


5 


63 


Czech Republic 


13 


5 


13 


Denmark 


71 


5 


71 


Estonia 


52 


5 


52 


Finland 


33 


5 


33 


France 


36 


5 


36 


Germany 


8 


5 


8 


Greece 


6 


5 


6 


Flong Kong-China 


33 


5 


33 


Flungary 


20 


5 


20 


Iceland 


19 


5 


19 


Indonesia 


48 


5 


48 


Ireland 


38 


5 


38 


Israel 


34 


5 


34 


Italy 


14 


5 


14 


Japan 


25 


5 


25 


Jordan 


47 


5 


47 


Korea 


30 


5 


30 


Lithuania 


45 


5 


45 


Luxembourg 


38 


5 


38 


Macao-China 


0 


4 


18 


Mexico 


22 


5 


22 


Montenegro 


0 


4 


13 


Netherlands 


27 


5 


27 


New Zealand 


26 


5 


26 


Norway 


47 


5 


47 


Poland 


9 


5 


9 


Portugal 


34 


5 


34 


Qatar 


63 


5 


63 


Romania 


5 


5 


5 


Serbia 


52 


5 


52 


Slovak Republic 


0 


4 


16 


Slovenia 


46 


5 


46 


Spain 


2 


5 


2 


Sweden 


5 


5 


5 


Switzerland 


0 


4 


2 


Tunisia 


20 


5 


20 


Turkey 


74 


5 


74 


United Kingdom 


8 


5 


8 


Uruguay 


33 


5 


33 
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Rankings 

13. Tables A2 and A3 in the Appendix present rankings for each country according to percent-correct 
scores obtained for distinct sets of items rated at the highest level of priority for inclusion by different 
countries. Table A2 gives rankings based on all PISA 2006 items, namely, those from previous cycles plus 
new items rated the highest priority for inclusion. Table A3 gives similar ranking based solely on new 
items developed for PISA 2006 and rated the highest priority for inclusion. Countries are sorted in 
alphabetical order. Note that while country rankings were produced for all countries participating in PISA, 
the list of countries which provided a basis for these rankings is shorter. It excludes countries which didn’t 
provide item ratings (Table A2) or those which rated only few items the highest priority (Sweden and 
Switzerland in Table A3). 

14. These results show quite a consistent ranking both across the sets of items and between the 
original ranking and the rankings based upon the various subsets of items. However, some countries’ 
ranking positions vary more than others. Rankings vary less when preferred items were used together with 
items from previous cycles, because in this case the pool of items from which percent-correct scores were 
obtained was much larger. In this case ranking positions of Lithuania, Korea, and Russian Federation vary 
more than for other countries, but even for these countries’ positions are remarkably stable. When only 
new PISA 2006 items are considered the ranks vary more. In this case ranking positions of Chinese Taipei, 
Korea, France, Croatia, and Japan, vary the most according to different sets of preferred items. In contrast, 
ranking positions of Finland or Canada do not change a lot. 

15. The results in Tables A2 and A3 are summarised in Table 2 below. Table 2 shows the initial 
report rank (OECD, 2007), the average percent-correct rank based upon all items, the average percent- 
correct rank based upon new PISA 2006 items, and finally the ranking based upon that country’s own 
highest rating items with and without previous cycles items considered. 

16. Country ranks based on their preferred items are usually very similar to those obtained for the full 
set of PISA 2006 science items - see Figures 1 and 2 below. Country ranks are also very similar to those 
presented in the initial report (OECD, 2007), obtained with a proper scaling methodology. 

17. There are, however, countries whose ranking position changes noticeably when only considering 
their preferred items. For example, the position of Macao-China or Lithuania increases by more than 5 
ranks, while the position of the Slovak Republic visibly decreases. However, these shifts are usually in a 
range which includes a sampling error. Moreover, these countries are usually in a group of moderate 
performers with very similar outcomes. In other words, moving up or down the ranking doesn’t mean that 
the mean performance of these countries on a final PISA scale would be noticeably different. In most 
cases, results would probably remain statistically similar to groups of countries with comparable 
performance levels. 

18. Tables A4 and A5 in the Appendix provide additional statistics. Generally, the median ranking 
position across different sets of countries’ preferred items is very similar to that obtained with a full set of 
items. Thus, on average countries have similar ranks to those presented in the initial report (OECD, 2007). 
The tables also provide a range of ranks according to different countries’ preferred items. 6 

19. The tables show that the average rank across rankings based on different countries’ preferred 
items is very close to the rank presented in the PISA 2006 initial report (OECD, 2007). While some 



r ’ This range was calculated by taking the 5% and 95% percentile of country’s positions across all rankings based on different countries sets of 
preferred items. 
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countries could gain from considering their preferred items it would be the opposite for others. There is no 
evidence that countries would generally gain from considering solely their preferred items. 

20. The variation in ranks across different sets of preferred items can be considered as partly random. 
There are only few countries which on average gain or lose when only some of the items are considered. 
The average position for most countries is very similar to the one based on all items, which means that the 
choice of preferred items is not systematically affecting their rank. However, variation in rankings suggests 
that having a bigger pool of items which compromises different countries’ or experts’ views is of crucial 
importance for the robustness of PISA rankings. 
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Table 2. Country rankings according to PISA 2006 initial report (OECD, 2007), percent-correct score, and 

preferred items percent-correct score 



Country 


PISA 2006 
initial report 
science 
literacy rank 


Percent-correct rank 
based on all PISA 
2006 items 


Rank on own 
preferred new PISA 
2006 items and items 
from previous cycles 


Percent-correct 
rank based on 
new PISA 2006 
items 


Rank on own 
preferred new 
PISA 2006 
items 


Finland 


1 


1 


1 


1 


1 


Flong Kong-China 


2 


2 


2 


2 


2 


Canada 


3 


3 


4 


3 


2 


Chinese Taipei 


4 


6 


7 


6 


8 


Estonia 


5 


4 


5 


4 


4 


Japan 


6 


5 


4 


5 


7 


New Zealand 


7 


7 


10 


7 


7 


Australia 


8 


8 


7 


8 


6 


Netherlands 


9 


9 


8 


9 


9 


Korea 


11 


11 


10 


11 


13 


Slovenia 


12 


13 


15 


13 


11 


Germany 


13 


12 


12 


12 


11 


United Kingdom 


14 


14 


14 


14 


13 


Czech Republic 


15 


15 


17 


15 


13 


Switzerland 


16 


16 


15 


16 




Macao-China 


17 


18 


11 


18 


12 


Austria 


18 


19 


22 


19 


17 


Belgium 


19 


17 


14 


17 


13 


Ireland 


20 


20 


18 


20 


13 


Flungary 


21 


22 


21 


22 


20 


Sweden 


22 


21 


21 


21 


30 


Poland 


23 


23 


24 


23 


27 


Denmark 


24 


24 


24 


24 


23 


France 


25 


25 


25 


25 


33 


Croatia 


26 


27 


28 


27 


29 


Iceland 


27 


26 


29 


26 


28 


Slovak Republic 


30 


31 


35 


31 


34 


Spain 


31 


28 


29 


28 




Lithuania 


32 


34 


28 


34 


31 


Norway 


33 


30 


28 


30 


27 


Luxembourg 


34 


35 


35 


35 


34 


Italy 


36 


36 


37 


36 


34 


Portugal 


37 


37 


37 


37 


35 


Greece 


38 


38 


33 


38 


37 


Israel 


39 


39 


39 


39 


39 


Chile 


40 


40 


41 


40 


41 


Serbia 


41 


43 


43 


43 


41 


Uruguay 


43 


41 


41 


41 


41 


Turkey 


44 


45 


45 


45 


44 


Jordan 


45 


44 


44 


44 


44 


Romania 


47 


48 


46 


48 


50 


Montenegro 


48 


50 


49 


50 


48 


Mexico 


49 


47 


48 


47 


49 


Indonesia 


50 


53 


52 


53 


52 


Colombia 


53 


51 


49 


51 


50 


Tunisia 


54 


54 


54 


54 


54 


Qatar 


56 


56 


56 


56 


56 
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Figure 1. Percent-correct ranking based on all PISA 2006 science items and the percent-correct ranking based 
on countries’ preferred items plus the items kept from the previous PISA cycles 
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Summary and Discussion 



21. Using an approximate (average percent-correct) methodology we have compared the ranks that 
would be obtained for PISA 2006 countries if the rankings had been derived from items judged by each 
country to be of highest priority for inclusion. 

22. The results show a remarkable consistency in the country rank orderings when taking the average 
position across rankings based on different countries’ preferred items and comparing it with the rank 
reported in the PISA 2006 initial report (OECD, 2007). This stability in average ranking position is highly 
desirable, but note that the variation of ranks across different sets of preferred items is non negligible. 

23. On average, only few countries systemically go up or down in their ranking position. Shifts in 
ranking positions for these countries are relatively small and comparable in magnitude to the sampling 
error (see Table 2.11c in the PISA 2006 initial report; OECD, 2007). For most of the countries such 
systematic shifts were not observed. 

24. Relatively high variation in countries’ ranks when different sets of items are considered suggests 
that the choice of items should compromise different experts’ and countries’ views. It also demonstrates 
the volatility of student assessment if the pool of test items is too narrow. 

25. Two lessons can be drawn from this analysis. First, the pool of test items has to be big enough to 
provide robust comparisons of countries and to accommodate diverse preferences. PISA uses around 100 
items for testing major domains and around 30 items for minor domains, which seems to be sufficient to 
limit the impact of individual countries’ views. Second, although this analysis suggests that final rankings 
can depend upon the choice of items, experts cannot identify items in advance that will advantage or 
disadvantage their country. This paper shows that this is the case in PISA, because countries in general do 
not gain or lose from considering their preferred items only. In other words, experts are not able to predict 
which items can increase their country’s chances of improving its ranking position in the final PISA test. 

26. Note also that these results should be considered as indicative only. A more rigorous analysis 
would be to use the full PISA scaling methodology and use a wider variety of approaches for selecting 
item subsets. For example these analysis items should be replicated using sub-groups of items based on 
other features of the items, e.g. item type, text length, continuous versus non-con tinuous, source language 
submission (see Lafontaine, Monseur, 2007, for an example of such analysis using PISA 2000 data). 
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Appendix 



Table A1. Preferred items statistics 



Country 


Items 

Rated 

Level 

Five 


Items rated at 
the highest 
level in the 
country 


Minimum 
rated level 


Maximum 
rated level 


Average 

rated 

level 


Comments 


OECD 














Australia 


50 


50 


1 


5 


4.4 




Austria 


14 


14 


1 


5 


3.8 




Belgium 


18 


18 


1 


5 


3.6 




Canada 


28 


28 


1 


5 


3.8 




Czech Republic 


13 


13 


1 


5 


3.5 




Denmark 


71 


71 


1 


5 


4.7 




Finland 


33 


33 


1 


5 


3.7 




France 


36 


36 


1 


5 


4.1 




Germany 


8 


8 


2 


5 


3.6 




Greece 


6 


6 


1 


5 


3.2 




Flungary 


20 


20 


1 


5 


4 




Iceland 


19 


19 


2 


5 


3.9 




Ireland 


38 


38 


1 


5 


4.1 




Italy 


14 


14 


2 


5 


3.7 




Japan 


25 


25 


2 


5 


4.2 




Korea 


30 


30 


1 


5 


4.1 




Luxembourg 


38 


38 


1 


5 


4 




Mexico 


22 


22 


2 


5 


4 




Netherlands 


27 


27 


1 


5 


3.7 




New Zealand 


26 


26 


1 


5 


4 




Norway 


47 


47 


1 


5 


4.2 




Poland 


9 


9 


1 


5 


3.7 




Portugal 


34 


34 


1 


5 


4 




Slovak Republic 


0 


16 


1 


4 


2.7 


Ranking for the items rated level 4 


Spain 


2 


2 


1 


5 


2.9 


Only two items were rated at the highest level 


Sweden 


5 


5 


1 


5 


3.7 




Switzerland 


0 


2 


2 


4 


2.5 


Only two items were rated at the highest level 


Turkey 


74 


74 


3 


5 


4.9 




United Kingdom 


8 


8 


1 


5 


3.5 




United States 












Not considered - Ranking not given 


Partner countries and economies 










Argentina 












Not considered - Ranking not given 


Azerbaijan 












Not considered - Ranking not given 


Brazil 












Not considered - Ranking not given 


Bulgaria 












Not considered - Ranking not given 


Chile 


22 


22 


2 


5 


4.1 




Chinese Taipei 


50 


50 


2 


5 


4.5 




Colombia 


58 


58 


2 


5 


4.7 




Croatia 


63 


63 


2 


5 


4.7 




Estonia 


52 


52 


1 


5 


4.5 




Flong Kong-China 


33 


33 


1 


5 


4.1 




Indonesia 


48 


48 


2 


5 


4.5 




Israel 


34 


34 


1 


5 


3.9 




Jordan 


47 


47 


2 


5 


4.3 




Kyrgyzstan 


81 


81 


5 


5 


5 


Not considered - Ranking given but all items included 


Latvia 












Not considered - Ranking not given 


Liechtenstein 












Not considered - Ranking not given 


Lithuania 


45 


45 


1 


5 


4.2 




Macao-China 


0 


18 


2 


4 


3.1 


Ranking for the items rated level 4 


Montenegro 


0 


13 


1 


4 


2.9 


Ranking for the items rated level 4 


Qatar 


63 


63 


1 


5 


4.7 




Romania 


5 


5 


1 


5 


2.5 




Russian Federation 












Ranking not given 


Serbia 


52 


52 


1 


5 


4.5 




Slovenia 


46 


46 


1 


5 


4.1 




Thailand 












Not considered - Ranking not given 


Tunisia 


20 


20 


1 


5 


3.9 




Uruguay 


33 


33 


1 


5 


4 
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Table A2 [Part 1/4], Country rankings according to other countries’ most preferred new PISA 2006 items and 

items from previous PISA cycles 



Country 



Other countries’ ranking positions according to the previous cycle items and a set of preferred new PISA 2006 items of: 

Chinese Czech 

Australia Austria Belgium Canada Chile T . . Colombia Croatia „ ... Denmark Estonia Finland 

M Taipei Republic 



50 



51 



51 



51 



50 



Argentina 



Australia 


7 


8 


9 


9 


8 


Austria 


20 


22 


21 


21 


20 


Azerbaijan 


55 


55 


55 


55 


55 


Belgium 


18 


18 


14 


15 


16 


Brazil 


53 


52 


53 


52 


54 


Bulgaria 


41 


40 


40 


42 


42 


Canada 


3 


5 


7 


4 


4 


Chile 


40 


41 


42 


40 


41 


Chinese Taipei 


6 


7 


6 


10 


7 


Colombia 


49 


49 


50 


49 


49 


Croatia 


28 


25 


26 


26 


27 


Czech Republic 


14 


16 


18 


18 


17 


Denmark 


24 


30 


24 


25 


24 


Estonia 


4 


4 


4 


5 


5 


Finland 


1 


1 


1 


1 


1 


France 


25 


21 


23 


24 


23 


Germany 


12 


12 


12 


12 


12 


Greece 


37 


31 


35 


36 


38 


Hong Kong-China 


2 


2 


2 


2 


2 


Hungary 


22 


23 


20 


22 


22 


Iceland 


27 


29 


28 


27 


28 


Indonesia 


52 


54 


54 


53 


52 


Ireland 


19 


20 


19 


19 


19 


Israel 


39 


39 


39 


39 


39 


Italy 


36 


37 


37 


38 


35 


Japan 


5 


3 


3 


3 


3 


Jordan 


45 


44 


44 


44 


44 


Korea 


10 


6 


5 


6 


6 


Kyrgyzstan 


57 


57 


57 


57 


57 


Latvia 


29 


35 


29 


31 


34 


Liechtenstein 


11 


10 


10 


8 


10 


Lithuania 


30 


28 


25 


32 


31 


Luxembourg 


35 


36 


33 


34 


32 


Macao-China 


17 


14 


13 


16 


14 


Mexico 


51 


47 


48 


47 


46 


Montenegro 


48 


50 


47 


50 


51 


Netherlands 


9 


11 


8 


7 


11 


New Zealand 


8 


9 


11 


11 


9 


Norway 


32 


32 


32 


30 


29 


Poland 


23 


24 


27 


23 


26 


Portugal 


38 


38 


38 


37 


37 


Qatar 


56 


56 


56 


56 


56 


Romania 


46 


45 


46 


46 


48 


Russian Federation 


26 


26 


30 


29 


33 


Serbia 


43 


43 


43 


43 


43 


Slovak Republic 


34 


34 


36 


35 


36 


Slovenia 


15 


15 


16 


13 


18 


Spain 


33 


27 


31 


28 


25 


Sweden 


21 


19 


22 


20 


21 


Switzerland 


16 


17 


15 


14 


15 


Thailand 


47 


48 


49 


48 


47 


Tunisia 


54 


53 


52 


54 


53 


Turkey 


44 


46 


45 


45 


45 


United Kingdom 


13 


13 


17 


17 


13 


United States 


31 


33 


34 


33 


30 


Uruguay 


42 


42 


41 


41 


40 



50 


50 


50 


52 


49 


49 


50 


6 


5 


8 


10 


7 


8 


10 


20 


19 


18 


20 


18 


19 


19 


55 


55 


55 


55 


55 


55 


55 


16 


18 


19 


16 


16 


17 


15 


52 


52 


52 


51 


52 


52 


52 


41 


42 


41 


40 


42 


42 


42 


3 


3 


3 


7 


3 


3 


4 


40 


40 


40 


42 


40 


41 


41 


7 


11 


5 


4 


9 


7 


5 


49 


49 


51 


49 


50 


50 


49 


27 


27 


28 


24 


27 


26 


28 


15 


14 


14 


17 


15 


13 


18 


24 


25 


24 


27 


24 


25 


25 


5 


6 


4 


5 


5 


5 


6 


1 


1 


1 


1 


1 


1 


1 


25 


24 


26 


22 


25 


24 


24 


13 


10 


12 


11 


11 


11 


12 


38 


38 


37 


36 


38 


37 


37 


2 


2 


2 


2 


2 


2 


2 


22 


22 


20 


25 


22 


21 


21 


26 


29 


25 


30 


26 


28 


27 


53 


54 


53 


54 


54 


54 


54 


17 


15 


22 


19 


20 


22 


20 


39 


39 


39 


38 


39 


39 


39 


36 


36 


36 


37 


36 


36 


36 


4 


7 


6 


3 


4 


4 


3 


44 


44 


44 


44 


44 


44 


44 


9 


13 


10 


6 


12 


12 


8 


57 


57 


57 


57 


57 


57 


57 


31 


31 


33 


33 


31 


33 


32 


11 


9 


9 


9 


10 


9 


7 


32 


35 


31 


28 


35 


34 


31 


35 


32 


35 


31 


34 


32 


34 


18 


20 


16 


13 


19 


16 


14 


47 


47 


47 


50 


48 


46 


46 


51 


51 


49 


48 


51 


51 


51 


10 


8 


11 


8 


8 


6 


9 


8 


4 


7 


12 


6 


10 


11 


34 


26 


29 


32 


29 


29 


29 


23 


23 


23 


23 


23 


23 


23 


37 


37 


38 


39 


37 


38 


38 


56 


56 


56 


56 


56 


56 


56 


48 


48 


48 


45 


47 


47 


47 


33 


34 


32 


26 


32 


30 


30 


43 


43 


43 


43 


43 


43 


43 


30 


33 


30 


34 


33 


31 


33 


14 


17 


13 


18 


14 


14 


16 


28 


28 


27 


29 


28 


27 


26 


21 


21 


21 


21 


21 


20 


22 


19 


16 


17 


15 


17 


18 


13 


46 


46 


46 


47 


46 


48 


48 


54 


53 


54 


53 


53 


53 


53 


45 


45 


45 


46 


45 


45 


45 


12 


12 


15 


14 


13 


15 


17 


29 


30 


34 


35 


30 


35 


35 


42 


41 


42 


41 


41 


40 


40 
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Table A2 [Part 2/4], Country rankings according to other countries’ most preferred new PISA 2006 items and 

items from previous PISA cycles 



Other countries' ranking positions according to the previous cycle items and a set of preferred new PISA 2006 items of: 



Country 


France 


Germany 


Greece 


Hong 

Kong- 

China 


Hungary 


Iceland 


Indonesia 


Ireland 


Israel 


Italy 


Japan 


Jordan 


Korea 


Lithuania 


Argentina 


49 


51 


51 


50 


50 


49 


51 


50 


51 


51 


50 


50 


50 


49 


Australia 


7 


6 


11 


9 


11 


10 


8 


9 


8 


10 


5 


10 


11 


9 


Austria 


20 


20 


20 


18 


19 


19 


19 


20 


19 


22 


20 


19 


19 


21 


Azerbaijan 


55 


55 


55 


55 


55 


55 


55 


55 


55 


55 


55 


55 


55 


55 


Belgium 


16 


16 


16 


19 


14 


16 


15 


14 


15 


17 


14 


16 


15 


15 


Brazil 


52 


53 


53 


53 


52 


52 


53 


52 


52 


54 


52 


52 


52 


52 


Bulgaria 


42 


40 


40 


42 


42 


41 


41 


42 


42 


40 


41 


42 


42 


42 


Canada 


4 


3 


6 


4 


6 


5 


3 


3 


4 


6 


3 


5 


4 


3 


Chile 


40 


42 


42 


41 


40 


42 


40 


41 


40 


42 


40 


41 


40 


41 


Chinese Taipei 


6 


10 


4 


5 


5 


6 


5 


10 


10 


5 


8 


8 


5 


5 


Colombia 


51 


49 


50 


49 


49 


51 


50 


49 


47 


49 


48 


49 


51 


50 


Croatia 


24 


27 


25 


27 


30 


28 


26 


25 


31 


27 


27 


26 


29 


30 


Czech Republic 


15 


17 


17 


15 


18 


14 


14 


19 


18 


14 


18 


13 


18 


14 


Denmark 


26 


25 


26 


23 


28 


26 


24 


26 


25 


25 


25 


24 


23 


24 


Estonia 


5 


4 


5 


6 


4 


4 


4 


4 


3 


4 


9 


3 


3 


6 


Finland 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


France 


25 


21 


24 


24 


22 


20 


25 


24 


23 


20 


22 


25 


25 


25 


Germany 


12 


12 


10 


11 


12 


11 


12 


11 


12 


11 


12 


11 


12 


12 


Greece 


37 


35 


33 


37 


36 


37 


36 


36 


36 


36 


36 


36 


37 


36 


Hong Kong-China 


2 


2 


2 


2 


2 


2 


2 


2 


2 


2 


2 


2 


2 


2 


Hungary 


22 


24 


23 


22 


21 


23 


22 


22 


24 


24 


23 


22 


22 


19 


Iceland 


27 


32 


28 


28 


26 


29 


27 


31 


29 


26 


29 


30 


27 


26 


Indonesia 


54 


52 


54 


54 


54 


53 


52 


54 


54 


53 


53 


54 


54 


54 


Ireland 


19 


18 


18 


20 


20 


22 


20 


18 


20 


21 


19 


21 


21 


22 


Israel 


39 


39 


39 


39 


39 


39 


39 


39 


39 


39 


38 


39 


38 


39 


Italy 


38 


37 


37 


36 


37 


34 


37 


37 


37 


37 


39 


37 


36 


37 


Japan 


3 


5 


3 


3 


3 


3 


7 


7 


5 


3 


4 


4 


6 


4 


Jordan 


44 


44 


43 


44 


44 


44 


44 


44 


44 


43 


45 


44 


44 


44 


Korea 


8 


11 


7 


13 


7 


8 


11 


13 


11 


7 


7 


12 


10 


10 


Kyrgyzstan 


57 


57 


57 


57 


57 


57 


57 


57 


57 


57 


57 


57 


57 


57 


Latvia 


30 


33 


34 


34 


29 


36 


29 


32 


32 


35 


32 


28 


30 


29 


Liechtenstein 


10 


7 


8 


8 


8 


7 


9 


6 


9 


8 


11 


7 


8 


8 


Lithuania 


33 


30 


30 


33 


25 


27 


33 


30 


30 


29 


30 


27 


26 


28 


Luxembourg 


35 


34 


32 


31 


35 


33 


35 


29 


33 


33 


35 


33 


31 


34 


Macao-China 


17 


14 


12 


14 


15 


12 


18 


16 


14 


13 


17 


18 


16 


13 


Mexico 


47 


46 


47 


47 


48 


45 


48 


45 


49 


46 


47 


48 


48 


46 


Montenegro 


50 


50 


48 


51 


51 


50 


49 


51 


50 


50 


51 


51 


49 


51 


Netherlands 


9 


9 


9 


7 


10 


9 


6 


5 


6 


9 


10 


6 


7 


7 


New Zealand 


11 


8 


13 


10 


9 


13 


10 


8 


7 


12 


6 


9 


9 


11 


Norway 


29 


26 


29 


26 


32 


30 


32 


28 


27 


28 


33 


29 


28 


27 


Poland 


23 


23 


22 


25 


24 


24 


23 


23 


22 


23 


24 


23 


24 


23 


Portugal 


36 


38 


38 


38 


38 


38 


38 


38 


38 


38 


37 


38 


39 


38 


Qatar 


56 


56 


56 


56 


56 


56 


56 


56 


56 


56 


56 


56 


56 


56 


Romania 


48 


45 


45 


45 


46 


47 


47 


46 


46 


45 


49 


47 


46 


48 


Russian Federation 


31 


29 


31 


30 


27 


31 


30 


35 


26 


31 


31 


32 


32 


33 


Serbia 


43 


43 


44 


43 


43 


43 


43 


43 


43 


44 


43 


43 


43 


43 


Slovak Republic 


32 


36 


36 


35 


34 


32 


31 


34 


35 


32 


34 


34 


35 


32 


Slovenia 


14 


19 


19 


21 


17 


17 


13 


15 


17 


19 


15 


14 


17 


16 


Spain 


28 


28 


27 


29 


31 


25 


28 


27 


28 


30 


26 


31 


34 


31 


Sweden 


21 


22 


21 


17 


23 


21 


21 


21 


21 


18 


21 


20 


20 


20 


Switzerland 


18 


15 


15 


16 


16 


15 


16 


17 


16 


16 


16 


15 


14 


18 


Thailand 


46 


48 


49 


48 


47 


48 


46 


48 


48 


48 


46 


46 


47 


47 


Tunisia 


53 


54 


52 


52 


53 


54 


54 


53 


53 


52 


54 


53 


53 


53 


Turkey 


45 


47 


46 


46 


45 


46 


45 


47 


45 


47 


44 


45 


45 


45 


United Kingdom 


13 


13 


14 


12 


13 


18 


17 


12 


13 


15 


13 


17 


13 


17 


United States 


34 


31 


35 


32 


33 


35 


34 


33 


34 


34 


28 


35 


33 


35 


Uruguay 


41 


41 


41 


40 


41 


40 


42 


40 


41 


41 


42 


40 


41 


40 
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Table A2 [Part 3/4], Country rankings according to other countries’ most preferred new PISA 2006 items and 

items from previous PISA cycles 



Other countries' ranking positions according to the previous cycle items and a set of preferred new PISA 2006 items of: 



Country 


Luxembourg 


Macao- 

China 


Mexico 


Montenegro 


Netherlands 


New 

Zealand 


Norway 


Poland 


Portugal 


Qatar 


Romania 


Argentina 


51 


51 


51 


51 


50 


49 


50 


50 


51 


49 


50 


Australia 


8 


7 


8 


9 


10 


11 


9 


8 


6 


7 


10 


Austria 


21 


20 


20 


21 


21 


19 


19 


22 


19 


20 


21 


Azerbaijan 


55 


55 


55 


55 


53 


55 


55 


55 


55 


55 


55 


Belgium 


16 


18 


15 


16 


18 


18 


18 


15 


14 


16 


17 


Brazil 


53 


52 


53 


52 


54 


52 


52 


53 


52 


52 


54 


Bulgaria 


41 


40 


41 


40 


40 


40 


40 


40 


41 


42 


40 


Canada 


4 


5 


3 


7 


6 


6 


5 


5 


4 


3 


7 


Chile 


40 


42 


40 


41 


41 


42 


42 


42 


40 


40 


42 


Chinese Taipei 


5 


3 


7 


3 


3 


5 


6 


7 


8 


5 


8 


Colombia 


50 


50 


49 


50 


51 


51 


51 


48 


49 


51 


48 


Croatia 


26 


30 


29 


26 


26 


27 


25 


28 


25 


27 


30 


Czech Republic 


15 


15 


17 


18 


15 


14 


16 


19 


17 


15 


16 


Denmark 


24 


24 


23 


27 


25 


26 


24 


26 


27 


25 


24 


Estonia 


3 


4 


4 


4 


7 


3 


3 


4 


5 


8 


4 


Finland 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


France 


29 


23 


25 


22 


29 


23 


27 


17 


23 


23 


25 


Germany 


12 


10 


12 


13 


14 


9 


12 


13 


12 


12 


11 


Greece 


36 


36 


36 


33 


36 


36 


37 


31 


36 


38 


32 


Hong Kong-China 


2 


2 


2 


2 


2 


2 


2 


2 


2 


2 


2 


Hungary 


22 


22 


19 


23 


22 


24 


22 


23 


22 


22 


22 


Iceland 


25 


31 


27 


29 


24 


30 


26 


30 


26 


26 


26 


Indonesia 


52 


54 


52 


54 


52 


54 


53 


52 


53 


53 


53 


Ireland 


19 


21 


21 


20 


20 


20 


21 


21 


21 


21 


19 


Israel 


39 


38 


38 


39 


39 


39 


39 


39 


39 


39 


39 


Italy 


37 


37 


37 


37 


37 


37 


36 


34 


38 


36 


37 


Japan 


6 


6 


5 


5 


4 


4 


4 


3 


3 


4 


3 


Jordan 


44 


44 


45 


44 


44 


43 


44 


44 


44 


44 


43 


Korea 


9 


14 


9 


6 


5 


12 


11 


6 


7 


11 


5 


Kyrgyzstan 


57 


57 


57 


57 


57 


57 


57 


57 


57 


57 


57 


Latvia 


30 


34 


31 


30 


33 


33 


32 


36 


32 


33 


33 


Liechtenstein 


11 


8 


11 


11 


9 


7 


10 


9 


11 


10 


6 


Lithuania 


27 


26 


26 


24 


32 


25 


31 


27 


33 


31 


29 


Luxembourg 


35 


35 


32 


35 


35 


34 


35 


35 


34 


34 


34 


Macao-China 


17 


11 


16 


10 


13 


13 


14 


12 


18 


18 


12 


Mexico 


48 


48 


48 


48 


47 


48 


47 


47 


46 


47 


45 


Montenegro 


49 


49 


50 


49 


49 


50 


49 


51 


50 


50 


51 


Netherlands 


7 


12 


6 


12 


8 


8 


7 


11 


9 


9 


9 


New Zealand 


10 


9 


10 


8 


11 


10 


8 


10 


10 


6 


13 


Norway 


31 


27 


28 


31 


30 


29 


28 


29 


28 


28 


28 


Poland 


23 


25 


24 


25 


23 


22 


23 


24 


24 


24 


23 


Portugal 


38 


39 


39 


38 


38 


38 


38 


38 


37 


37 


38 


Qatar 


56 


56 


56 


56 


56 


56 


56 


56 


56 


56 


56 


Romania 


46 


47 


46 


46 


46 


45 


46 


46 


47 


48 


46 


Russian Federation 


34 


28 


35 


28 


27 


28 


29 


32 


30 


35 


31 


Serbia 


43 


43 


43 


43 


42 


44 


43 


43 


43 


43 


44 


Slovak Republic 


32 


33 


34 


36 


31 


32 


34 


37 


35 


32 


36 


Slovenia 


13 


17 


18 


14 


12 


16 


15 


18 


13 


14 


18 


Spain 


28 


29 


30 


32 


28 


31 


30 


25 


29 


29 


27 


Sweden 


20 


19 


22 


19 


19 


21 


20 


20 


20 


19 


20 


Switzerland 


18 


16 


14 


17 


16 


17 


17 


16 


15 


17 


15 


Thailand 


47 


45 


47 


47 


48 


46 


48 


49 


48 


46 


49 


Tunisia 


54 


53 


54 


53 


55 


53 


54 


54 


54 


54 


52 


Turkey 


45 


46 


44 


45 


45 


47 


45 


45 


45 


45 


47 


United Kingdom 


14 


13 


13 


15 


17 


15 


13 


14 


16 


13 


14 


United States 


33 


32 


33 


34 


34 


35 


33 


33 


31 


30 


35 


Uruguay 


42 


41 


42 


42 


43 


41 


41 


41 


42 


41 


41 
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Table A2 [Part 4/4], Country rankings according to other countries’ most preferred new PISA 2006 items and 

items from previous PISA cycles 



Other countries' ranking positions according to the previous cycle items and a set of preferred new PISA 2006 items of: 



Country 


Serbia 


Slovak 

Republic 


Slovenia 


Spain 


Sweden 


Switzerland 


Tunisia 


Turkey 


United 

Kingdom 


Uruguay 


Argentina 


51 


50 


49 


53 


51 


51 


51 


49 


51 


49 


Australia 


9 


8 


7 


11 


10 


10 


11 


7 


8 


7 


Austria 


19 


18 


20 


21 


20 


21 


19 


18 


24 


19 


Azerbaijan 


55 


55 


55 


55 


55 


55 


55 


55 


55 


55 


Belgium 


18 


15 


17 


16 


17 


16 


16 


17 


13 


14 


Brazil 


52 


52 


52 


54 


53 


54 


52 


52 


52 


52 


Bulgaria 


41 


40 


42 


40 


40 


40 


42 


42 


40 


42 


Canada 


3 


5 


3 


8 


7 


8 


7 


3 


4 


5 


Chile 


40 


41 


40 


42 


42 


42 


40 


40 


42 


40 


Chinese Taipei 


6 


3 


6 


3 


4 


5 


4 


6 


6 


3 


Colombia 


50 


48 


51 


47 


47 


49 


50 


50 


48 


51 


Croatia 


28 


33 


26 


27 


25 


29 


28 


26 


25 


27 


Czech Republic 


15 


17 


14 


17 


16 


17 


14 


15 


18 


15 


Denmark 


24 


23 


25 


26 


26 


25 


27 


24 


26 


25 


Estonia 


5 


6 


5 


6 


5 


7 


5 


5 


7 


6 


Finland 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


France 


26 


24 


24 


23 


22 


22 


24 


25 


20 


20 


Germany 


12 


12 


12 


10 


11 


11 


10 


11 


12 


12 


Greece 


37 


36 


38 


30 


35 


32 


35 


37 


36 


36 


Hong Kong-China 


2 


2 


2 


2 


2 


2 


2 


2 


2 


2 


Hungary 


22 


22 


22 


22 


24 


23 


20 


22 


21 


23 


Iceland 


27 


28 


28 


32 


30 


26 


29 


27 


29 


30 


Indonesia 


53 


54 


54 


50 


54 


53 


53 


53 


54 


53 


Ireland 


20 


21 


19 


19 


18 


19 


22 


20 


19 


22 


Israel 


39 


39 


39 


37 


38 


38 


39 


39 


39 


39 


Italy 


36 


37 


36 


38 


37 


37 


37 


36 


37 


37 


Japan 


4 


4 


4 


4 


3 


3 


3 


4 


3 


4 


Jordan 


44 


44 


44 


43 


43 


43 


44 


44 


44 


44 


Korea 


10 


7 


11 


5 


6 


4 


6 


12 


5 


11 


Kyrgyzstan 


57 


57 


57 


57 


57 


57 


57 


57 


57 


57 


Latvia 


32 


29 


35 


33 


34 


33 


32 


30 


33 


32 


Liechtenstein 


8 


9 


10 


7 


8 


6 


8 


10 


9 


10 


Lithuania 


34 


26 


32 


25 


28 


27 


25 


33 


28 


26 


Luxembourg 


35 


34 


33 


36 


31 


35 


34 


35 


34 


35 


Macao-China 


16 


13 


16 


12 


13 


12 


13 


19 


17 


13 


Mexico 


48 


49 


46 


48 


48 


48 


48 


47 


49 


46 


Montenegro 


49 


51 


50 


51 


49 


52 


49 


51 


50 


50 


Netherlands 


11 


11 


8 


9 


9 


9 


9 


9 


10 


8 


New Zealand 


7 


10 


9 


13 


12 


13 


12 


8 


11 


9 


Norway 


29 


27 


29 


31 


32 


30 


31 


29 


30 


31 


Poland 


23 


25 


23 


24 


23 


24 


23 


23 


23 


24 


Portugal 


38 


38 


37 


39 


39 


39 


38 


38 


38 


38 


Qatar 


56 


56 


56 


56 


56 


56 


56 


56 


56 


56 


Romania 


46 


47 


47 


45 


45 


45 


47 


48 


47 


47 


Russian Federation 


25 


30 


34 


28 


29 


31 


26 


34 


31 


29 


Serbia 


43 


43 


43 


44 


44 


44 


43 


43 


43 


43 


Slovak Republic 


30 


35 


31 


35 


36 


36 


33 


32 


35 


33 


Slovenia 


13 


20 


15 


18 


19 


18 


18 


14 


15 


17 


Spain 


31 


31 


27 


29 


27 


28 


30 


28 


27 


28 


Sweden 


21 


19 


21 


20 


21 


20 


21 


21 


22 


21 


Switzerland 


17 


14 


18 


14 


15 


15 


15 


16 


16 


18 


Thailand 


47 


46 


48 


49 


50 


47 


46 


46 


46 


48 


Tunisia 


54 


53 


53 


52 


52 


50 


54 


54 


53 


54 


Turkey 


45 


45 


45 


46 


46 


46 


45 


45 


45 


45 


United Kingdom 


14 


16 


13 


15 


14 


14 


17 


13 


14 


16 


United States 


33 


32 


30 


34 


33 


34 


36 


31 


32 


34 


Uruguay 


42 


42 


41 


41 


41 


41 


41 


41 


41 


41 
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Table A3 [Part 1/4]. Country rankings according to other countries’ most preferred new PISA 2006 items 



Country 


Australia 


Other countries' ranking positions according to the set of preferred new PISA 2006 items of: 

Phinpc;p P7prh 

Austria Belgium Canada Chile T . . Colombia Croatia „ ... Denmark Estonia 

M Taipei Republic 


Finland 


France 


Argentina 


49 


50 


50 


51 


48 


49 


48 


50 


51 


49 


49 


49 


49 


Australia 


6 


6 


6 


7 


5 


5 


5 


7 


7 


5 


8 


8 


4 


Austria 


16 


17 


19 


18 


16 


17 


13 


15 


16 


15 


14 


18 


17 


Azerbaijan 


55 


53 


55 


55 


55 


55 


55 


55 


55 


55 


55 


55 


55 


Belgium 


19 


20 


13 


12 


20 


18 


18 


20 


19 


16 


17 


15 


18 


Brazil 


53 


51 


53 


52 


54 


52 


52 


52 


49 


52 


52 


52 


52 


Bulgaria 


42 


42 


43 


43 


43 


43 


43 


41 


41 


43 


43 


43 


43 


Canada 


2 


2 


3 


2 


2 


2 


2 


3 


5 


2 


3 


3 


2 


Chile 


40 


40 


42 


40 


41 


40 


40 


40 


42 


40 


40 


41 


39 


Chinese Taipei 


9 


8 


9 


13 


10 


8 


15 


5 


3 


9 


9 


6 


8 


Colombia 


50 


52 


51 


50 


49 


50 


50 


51 


50 


51 


51 


51 


51 


Croatia 


28 


26 


25 


24 


25 


27 


28 


29 


17 


30 


24 


29 


23 


Czech Republic 


10 


12 


20 


17 


17 


13 


12 


12 


13 


14 


11 


14 


15 


Denmark 


23 


38 


24 


27 


26 


28 


24 


24 


30 


23 


25 


26 


27 


Estonia 


4 


5 


4 


6 


6 


7 


6 


4 


9 


6 


4 


4 


5 


Finland 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


France 


30 


21 


29 


29 


23 


30 


25 


33 


27 


26 


26 


25 


33 


Germany 


11 


16 


12 


15 


11 


14 


10 


11 


12 


11 


10 


16 


12 


Greece 


38 


33 


36 


37 


38 


38 


38 


38 


39 


38 


38 


38 


38 


Hong Kong-China 


3 


11 


2 


3 


3 


3 


4 


2 


2 


3 


2 


2 


3 


Hungary 


22 


18 


14 


22 


21 


21 


21 


19 


29 


21 


18 


20 


21 


Iceland 


26 


29 


26 


23 


29 


24 


31 


26 


28 


25 


29 


24 


25 


Indonesia 


52 


54 


54 


53 


52 


53 


54 


53 


54 


54 


54 


53 


54 


Ireland 


17 


24 


17 


19 


18 


16 


14 


22 


20 


19 


22 


21 


16 


Israel 


39 


39 


39 


39 


39 


39 


39 


39 


38 


39 


39 


39 


40 


Italy 


36 


32 


38 


38 


33 


37 


36 


37 


36 


37 


35 


37 


37 


Japan 


7 


3 


10 


8 


7 


6 


9 


8 


4 


8 


7 


10 


9 


Jordan 


46 


44 


44 


44 


47 


45 


46 


44 


47 


44 


44 


46 


44 


Korea 


13 


9 


8 


10 


9 


12 


20 


14 


11 


17 


21 


11 


14 


Kyrgyzstan 


57 


57 


56 


57 


57 


57 


57 


57 


57 


57 


57 


57 


57 


Latvia 


27 


37 


23 


30 


36 


29 


32 


27 


34 


28 


34 


33 


29 


Liechtenstein 


15 


15 


18 


11 


13 


15 


11 


13 


10 


12 


12 


7 


11 


Lithuania 


32 


35 


27 


36 


37 


34 


35 


34 


32 


35 


37 


35 


36 


Luxembourg 


34 


36 


30 


33 


30 


35 


33 


35 


25 


33 


30 


32 


35 


Macao-China 


21 


23 


15 


21 


14 


22 


22 


18 


21 


22 


20 


19 


20 


Mexico 


51 


47 


49 


47 


45 


47 


47 


47 


52 


47 


46 


45 


47 


Montenegro 


48 


49 


46 


49 


51 


51 


51 


49 


46 


50 


50 


50 


50 


Netherlands 


8 


13 


5 


4 


12 


9 


7 


10 


6 


7 


5 


9 


7 


New Zealand 


5 


4 


7 


5 


4 


4 


3 


6 


8 


4 


6 


5 


6 


Norway 


33 


34 


35 


34 


31 


33 


26 


31 


35 


31 


31 


31 


31 


Poland 


24 


27 


31 


25 


32 


23 


23 


23 


23 


24 


23 


23 


24 


Portugal 


37 


31 


37 


35 


28 


36 


37 


36 


37 


36 


36 


36 


30 


Qatar 


56 


56 


57 


56 


56 


56 


56 


56 


56 


56 


56 


56 


56 


Romania 


47 


45 


48 


48 


50 


48 


49 


48 


44 


48 


48 


48 


48 


Russian Federation 


25 


22 


28 


26 


35 


32 


34 


30 


22 


34 


32 


30 


34 


Serbia 


41 


41 


40 


41 


42 


41 


42 


42 


40 


42 


42 


42 


41 


Slovak Republic 


31 


19 


32 


32 


34 


25 


30 


25 


26 


29 


28 


28 


26 


Slovenia 


14 


7 


11 


9 


15 


10 


16 


9 


15 


13 


13 


12 


10 


Spain 


35 


30 


34 


28 


27 


31 


29 


28 


31 


32 


27 


27 


28 


Sweden 


20 


14 


22 


20 


22 


20 


19 


21 


24 


20 


16 


22 


22 


Switzerland 


18 


25 


16 


14 


19 


19 


17 


17 


18 


18 


19 


13 


19 


Thailand 


45 


48 


47 


46 


46 


46 


44 


46 


48 


46 


47 


47 


46 


Tunisia 


54 


55 


52 


54 


53 


54 


53 


54 


53 


53 


53 


54 


53 


Turkey 


44 


46 


45 


45 


44 


44 


45 


45 


45 


45 


45 


44 


45 


United Kingdom 


12 


10 


21 


16 


8 


11 


8 


16 


14 


10 


15 


17 


13 


United States 


29 


28 


33 


31 


24 


26 


27 


32 


33 


27 


33 


34 


32 


Uruguay 


43 


43 


41 


42 


40 


42 


41 


43 


43 


41 


41 


40 


42 
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Table A3 [Part 2/4]. Country rankings according to other countries’ most preferred new PISA 2006 items 



Country 


Germany 


Other countries' ranking positions according to the set of preferred new PISA 2006 items of: 

Hong 

Greece Kong- Hungary Iceland Indonesia Ireland Israel Italy Japan Jordan Korea 

China 


Lithuania 


Argentina 


50 


50 


48 


50 


46 


50 


49 


51 


51 


49 


49 


49 


49 


Australia 


4 


13 


9 


7 


12 


7 


7 


7 


12 


5 


9 


8 


8 


Austria 


9 


9 


15 


13 


17 


14 


14 


16 


20 


19 


14 


14 


18 


Azerbaijan 


54 


53 


55 


57 


55 


55 


55 


55 


45 


55 


55 


55 


55 


Belgium 


14 


22 


21 


16 


15 


15 


16 


15 


19 


13 


15 


17 


15 


Brazil 


53 


51 


53 


51 


52 


53 


52 


52 


55 


51 


52 


52 


52 


Bulgaria 


43 


42 


43 


42 


42 


42 


44 


43 


42 


43 


43 


43 


43 


Canada 


3 


4 


3 


4 


3 


2 


3 


4 


3 


2 


5 


3 


3 


Chile 


42 


44 


40 


39 


43 


40 


40 


40 


41 


40 


42 


40 


41 


Chinese Taipei 


33 


6 


8 


9 


6 


6 


15 


12 


7 


11 


11 


7 


5 


Colombia 


52 


54 


50 


49 


51 


51 


50 


48 


53 


47 


51 


51 


51 


Croatia 


22 


11 


26 


31 


27 


26 


24 


34 


24 


27 


26 


29 


30 


Czech Republic 


7 


14 


13 


17 


5 


12 


17 


17 


5 


17 


13 


19 


11 


Denmark 


25 


28 


23 


30 


30 


24 


28 


25 


26 


28 


24 


22 


24 


Estonia 


6 


5 


6 


6 


4 


5 


6 


3 


4 


9 


3 


4 


6 


Finland 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


France 


20 


36 


27 


24 


21 


31 


27 


24 


21 


22 


27 


32 


29 


Germany 


11 


7 


10 


11 


14 


13 


9 


10 


16 


15 


10 


11 


13 


Greece 


40 


37 


38 


38 


38 


38 


38 


38 


38 


38 


38 


39 


38 


Hong Kong-China 


10 


2 


2 


5 


2 


4 


2 


2 


2 


4 


2 


2 


2 


Hungary 


30 


26 


18 


20 


22 


20 


22 


23 


30 


24 


21 


23 


16 


Iceland 


34 


25 


28 


23 


28 


25 


32 


30 


22 


29 


28 


24 


25 


Indonesia 


51 


55 


54 


53 


53 


52 


53 


54 


52 


53 


54 


54 


54 


Ireland 


12 


15 


17 


19 


26 


21 


13 


20 


23 


18 


22 


20 


22 


Israel 


39 


39 


39 


40 


39 


39 


39 


39 


39 


37 


39 


38 


39 


Italy 


35 


34 


35 


37 


29 


37 


37 


37 


34 


39 


36 


36 


36 


Japan 


29 


20 


7 


3 


10 


11 


12 


9 


10 


7 


8 


10 


9 


Jordan 


47 


43 


44 


48 


44 


45 


43 


45 


44 


48 


44 


47 


44 


Korea 


36 


24 


22 


12 


19 


16 


20 


14 


18 


12 


20 


13 


14 


Kyrgyzstan 


57 


57 


57 


56 


57 


57 


57 


57 


57 


57 


57 


57 


57 


Latvia 


31 


35 


34 


22 


37 


28 


33 


28 


37 


31 


25 


27 


26 


Liechtenstein 


17 


10 


11 


10 


8 


10 


8 


8 


11 


14 


7 


9 


10 


Lithuania 


37 


38 


36 


28 


33 


35 


34 


35 


35 


35 


34 


30 


31 


Luxembourg 


32 


23 


29 


35 


31 


34 


26 


27 


33 


33 


31 


28 


32 


Macao-China 


21 


19 


16 


21 


13 


22 


19 


19 


14 


20 


19 


21 


19 


Mexico 


46 


48 


45 


47 


45 


49 


46 


50 


48 


46 


48 


50 


47 


Montenegro 


45 


46 


51 


52 


49 


48 


51 


49 


49 


52 


50 


48 


50 


Netherlands 


8 


3 


4 


8 


7 


3 


5 


6 


6 


6 


4 


6 


4 


New Zealand 


2 


12 


5 


2 


11 


8 


4 


5 


9 


3 


6 


5 


7 


Norway 


19 


29 


24 


34 


34 


33 


30 


29 


27 


36 


30 


26 


28 


Poland 


13 


21 


25 


27 


20 


23 


23 


22 


28 


26 


23 


25 


23 


Portugal 


28 


33 


37 


36 


35 


36 


36 


36 


36 


32 


37 


37 


37 


Qatar 


56 


56 


56 


55 


56 


56 


56 


56 


56 


56 


56 


56 


56 


Romania 


49 


45 


47 


46 


50 


47 


48 


46 


46 


50 


47 


46 


48 


Russian Federation 


26 


32 


33 


25 


32 


29 


35 


26 


29 


34 


32 


34 


35 


Serbia 


38 


41 


42 


44 


41 


41 


42 


42 


40 


42 


40 


42 


42 


Slovak Republic 


23 


27 


31 


29 


25 


27 


29 


33 


25 


30 


29 


33 


27 


Slovenia 


16 


17 


20 


14 


9 


9 


11 


13 


13 


8 


12 


16 


12 


Spain 


27 


30 


32 


33 


23 


30 


25 


31 


31 


25 


33 


35 


33 


Sweden 


24 


18 


14 


26 


24 


19 


21 


21 


8 


21 


18 


18 


21 


Switzerland 


15 


16 


19 


18 


16 


17 


18 


18 


17 


16 


17 


15 


20 


Thailand 


44 


49 


49 


43 


47 


46 


45 


47 


47 


45 


46 


45 


46 


Tunisia 


55 


52 


52 


54 


54 


54 


54 


53 


54 


54 


53 


53 


53 


Turkey 


48 


47 


46 


45 


48 


44 


47 


44 


50 


44 


45 


44 


45 


United Kingdom 


5 


8 


12 


15 


18 


18 


10 


11 


15 


10 


16 


12 


17 


United States 


18 


31 


30 


32 


36 


32 


31 


32 


32 


23 


35 


31 


34 


Uruguay 


41 


40 


41 


41 


40 


43 


41 


41 


43 


41 


41 


41 


40 
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Table A3 [Part 3/4]. Country rankings according to other countries’ most preferred new PISA 2006 items 



Country 


Luxembourg 


Other countries' ranking positions according to the set of preferred new PISA 2006 items of: 
^hina Mexico Montenegro Netherlands Zealand Norwa y Poland Portugal 


Qatar 


Romania 


Argentina 


50 


48 


50 


50 


48 


49 


45 


51 


49 


49 


50 


Australia 


6 


6 


5 


8 


15 


9 


5 


5 


5 


16 


6 


Austria 


19 


11 


16 


19 


13 


15 


24 


17 


18 


15 


19 


Azerbaijan 


55 


55 


55 


53 


55 


54 


51 


55 


55 


44 


55 


Belgium 


15 


20 


17 


15 


17 


20 


12 


14 


15 


27 


15 


Brazil 


53 


52 


53 


51 


51 


52 


53 


52 


52 


54 


53 


Bulgaria 


42 


43 


43 


41 


41 


43 


43 


43 


43 


41 


42 


Canada 


2 


3 


2 


5 


4 


4 


2 


2 


3 


3 


2 


Chile 


40 


40 


40 


40 


40 


40 


42 


40 


40 


40 


40 


Chinese Taipei 


8 


7 


12 


2 


5 


6 


17 


11 


6 


31 


8 


Colombia 


51 


51 


51 


52 


52 


51 


52 


50 


51 


47 


51 


Croatia 


28 


35 


32 


18 


26 


25 


31 


25 


27 


29 


28 


Czech Republic 


12 


9 


13 


17 


8 


12 


16 


15 


13 


5 


12 


Denmark 


24 


21 


23 


29 


29 


23 


35 


27 


23 


20 


24 


Estonia 


4 


5 


7 


3 


3 


2 


7 


8 


9 


6 


4 


Finland 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


1 


France 


35 


27 


34 


23 


25 


35 


6 


24 


25 


38 


35 


Germany 


14 


10 


14 


14 


9 


13 


19 


13 


12 


17 


14 


Greece 


36 


37 


38 


37 


38 


38 


33 


38 


38 


35 


36 


Hong Kong-China 


3 


2 


3 


6 


2 


3 


4 


6 


2 


2 


3 


Hungary 


20 


18 


10 


24 


23 


21 


21 


20 


22 


14 


20 


Iceland 


25 


32 


25 


31 


31 


26 


28 


23 


26 


9 


25 


Indonesia 


52 


54 


52 


55 


53 


53 


50 


53 


53 


53 


52 


Ireland 


18 


23 


20 


27 


20 


22 


22 


22 


21 


19 


18 


Israel 


39 


38 


39 


39 


39 


39 


39 


39 


39 


46 


39 


Italy 


38 


34 


36 


38 


37 


36 


26 


37 


37 


37 


38 


Japan 


10 


22 


9 


11 


6 


7 


14 


3 


8 


21 


10 


Jordan 


45 


45 


48 


43 


44 


44 


44 


45 


44 


39 


45 


Korea 


11 


36 


19 


10 


22 


16 


15 


10 


16 


26 


11 


Kyrgyzstan 


56 


57 


56 


56 


57 


57 


57 


57 


57 


57 


56 


Latvia 


29 


33 


26 


26 


32 


28 


38 


30 


33 


33 


29 


Liechtenstein 


16 


15 


15 


22 


12 


11 


20 


12 


10 


4 


16 


Lithuania 


31 


24 


31 


25 


28 


34 


36 


36 


34 


36 


31 


Luxembourg 


34 


31 


28 


34 


34 


33 


37 


34 


32 


34 


34 


Macao-China 


21 


12 


22 


9 


16 


19 


9 


21 


20 


11 


21 


Mexico 


49 


50 


49 


49 


47 


50 


47 


47 


47 


43 


49 


Montenegro 


48 


47 


47 


48 


50 


47 


49 


49 


50 


51 


48 


Netherlands 


5 


16 


4 


16 


11 


8 


13 


7 


7 


12 


5 


New Zealand 


7 


4 


6 


4 


7 


5 


3 


4 


4 


8 


7 


Norway 


32 


17 


27 


33 


30 


27 


29 


29 


28 


22 


32 


Poland 


23 


26 


24 


28 


21 


24 


27 


26 


24 


18 


23 


Portugal 


37 


39 


37 


35 


36 


37 


32 


35 


36 


28 


37 


Qatar 


57 


56 


57 


57 


56 


56 


56 


56 


56 


56 


57 


Romania 


47 


49 


46 


47 


45 


48 


54 


48 


48 


50 


47 


Russian Federation 


33 


29 


35 


20 


27 


29 


34 


31 


35 


32 


33 


Serbia 


41 


41 


41 


42 


43 


41 


41 


41 


42 


42 


41 


Slovak Republic 


26 


25 


30 


30 


24 


30 


30 


32 


30 


25 


26 


Slovenia 


9 


13 


11 


7 


10 


10 


10 


9 


11 


7 


9 


Spain 


30 


30 


33 


36 


33 


31 


18 


33 


31 


24 


30 


Sweden 


17 


14 


21 


12 


19 


17 


11 


19 


19 


13 


17 


Switzerland 


22 


19 


18 


21 


18 


18 


23 


16 


17 


23 


22 


Thailand 


46 


44 


45 


46 


46 


46 


46 


46 


46 


52 


46 


Tunisia 


54 


53 


54 


54 


54 


55 


55 


54 


54 


55 


54 


Turkey 


44 


46 


44 


45 


49 


45 


48 


44 


45 


48 


44 


United Kingdom 


13 


8 


8 


13 


14 


14 


8 


18 


14 


10 


13 


United States 


27 


28 


29 


32 


35 


32 


25 


28 


29 


30 


27 


Uruguay 


43 


42 


42 


44 


42 


42 


40 


42 


41 


45 


43 
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Table A3 [Part 4/4]. Country rankings according to other countries’ most preferred new PISA 2006 items 



Country 


Serbia 


Other countries' ranking positions according to the set of preferred new PISA 2006 items of: 

Slovak oi ■ „ , t ■ ■ t i United 

,, ... Slovenia Sweden Tunisia Turkey ... . 

Republic Kingdom 


Uruguay 


Argentina 


50 


48 


50 


48 


50 


49 


49 


48 


Australia 


6 


4 


5 


6 


13 


5 


3 


8 


Austria 


16 


14 


15 


13 


14 


15 


21 


15 


Azerbaijan 


53 


55 


53 


50 


55 


55 


54 


55 


Belgium 


19 


15 


18 


23 


17 


17 


9 


14 


Brazil 


52 


52 


52 


54 


52 


52 


47 


52 


Bulgaria 


42 


41 


43 


41 


46 


43 


42 


42 


Canada 


2 


2 


3 


3 


5 


2 


4 


3 


Chile 


40 


39 


39 


42 


40 


40 


41 


40 


Chinese Taipei 


8 


3 


10 


8 


2 


8 


25 


4 


Colombia 


51 


47 


51 


49 


53 


51 


50 


51 


Croatia 


28 


36 


25 


15 


30 


26 


6 


25 


Czech Republic 


12 


18 


9 


5 


9 


14 


12 


11 


Denmark 


26 


21 


24 


33 


32 


24 


33 


27 


Estonia 


5 


7 


6 


7 


4 


6 


17 


6 


Finland 


1 


1 


1 


1 


1 


1 


1 


1 


France 


32 


31 


26 


28 


28 


25 


15 


20 


Germany 


14 


12 


14 


11 


10 


10 


22 


10 


Greece 


38 


38 


38 


46 


37 


38 


39 


38 


Hong Kong-China 


3 


6 


2 


2 


6 


3 


23 


2 


Hungary 


22 


20 


19 


32 


19 


20 


10 


22 


Iceland 


27 


25 


29 


34 


29 


27 


29 


33 


Indonesia 


54 


53 


54 


53 


51 


53 


53 


53 


Ireland 


17 


23 


17 


19 


25 


19 


14 


23 


Israel 


39 


40 


40 


37 


39 


39 


38 


39 


Italy 


37 


37 


35 


27 


36 


36 


30 


36 


Japan 


7 


8 


8 


4 


3 


9 


8 


7 


Jordan 


44 


46 


44 


43 


43 


45 


48 


44 


Korea 


15 


10 


16 


16 


12 


16 


18 


19 


Kyrgyzstan 


57 


56 


57 


57 


57 


57 


57 


57 


Latvia 


30 


24 


33 


35 


27 


30 


35 


32 


Liechtenstein 


10 


9 


12 


17 


8 


11 


16 


13 


Lithuania 


35 


29 


37 


36 


26 


35 


37 


29 


Luxembourg 


34 


33 


32 


21 


33 


33 


36 


35 


Macao-China 


20 


17 


20 


24 


16 


21 


34 


17 


Mexico 


49 


51 


47 


52 


47 


47 


51 


47 


Montenegro 


48 


50 


49 


44 


48 


50 


46 


49 


Netherlands 


11 


11 


7 


9 


7 


7 


7 


9 


New Zealand 


4 


5 


4 


12 


11 


4 


2 


5 


Norway 


31 


26 


31 


38 


31 


32 


31 


34 


Poland 


23 


27 


23 


18 


21 


23 


11 


24 


Portugal 


36 


35 


34 


31 


38 


37 


26 


37 


Qatar 


56 


57 


56 


56 


56 


56 


56 


56 


Romania 


47 


49 


48 


47 


49 


48 


52 


50 


Russian Federation 


24 


30 


36 


22 


24 


34 


32 


31 


Serbia 


41 


45 


42 


39 


41 


42 


40 


43 


Slovak Republic 


25 


34 


27 


25 


23 


28 


27 


28 


Slovenia 


9 


22 


11 


20 


15 


12 


5 


12 


Spain 


33 


32 


30 


26 


34 


31 


28 


26 


Sweden 


21 


19 


21 


30 


22 


22 


24 


21 


Switzerland 


18 


13 


22 


14 


18 


18 


20 


18 


Thailand 


46 


42 


46 


51 


44 


46 


45 


46 


Tunisia 


55 


54 


55 


55 


54 


54 


55 


54 


Turkey 


45 


43 


45 


45 


45 


44 


43 


45 


United Kingdom 


13 


16 


13 


10 


20 


13 


13 


16 


United States 


29 


28 


28 


29 


35 


29 


19 


30 


Uruguay 


43 


44 


41 


40 


42 


41 


44 


41 
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Table A4. Rankings using all PISA 2006 items (items from previous cycles and new PISA 2006 items) 



Country 


Initial 

report 

science 

literacy 

rank 


Percent 
correct 
ranking 
based on all 
PISA 2006 
items 


Rankings based on countries’ preferred 
items and previous PISA cycles items 

Median Range. Range: 

. lower 3 . 

rank rank upper rank 


Rank on own 
preferred new 
PISA 2006 
items and 
previous PISA 
cycles items 


Difference 
between initial 
report rank 
and preferred 
items rank 


Difference between 
percent-correct rank 
based on all items 
and rank based on 
preferred new PISA 
2006 items 


Finland 


1 


1 


1 


1 


1 


1 


0 


0 


Flong Kong-China 


2 


2 


2 


2 


2 


2 


0 


0 


Canada 


3 


3 


4 


3 


7 


4 


-1 


-1 


Chinese Taipei 


4 


6 


6 


3 


10 


7 


-3 


-1 


Estonia 


5 


4 


5 


3 


7 


5 


0 


-1 


Japan 


6 


5 


4 


3 


7 


4 


2 


1 


New Zealand 


7 


7 


10 


6 


13 


10 


-3 


-3 


Australia 


8 


8 


8 


6 


11 


7 


1 


1 


Netherlands 


9 


9 


9 


6 


11 


8 


1 


1 


Liechtenstein 


10 


10 


9 


6 


11 








Korea 


11 


11 


9 


5 


13 


10 


1 


1 


Slovenia 


12 


13 


16 


13 


19 


15 


-3 


-2 


Germany 


13 


12 


12 


10 


13 


12 


1 


0 


United Kingdom 


14 


14 


14 


12 


17 


14 


0 


0 


Czech Republic 


15 


15 


16 


14 


18 


17 


-2 


-2 


Switzerland 


16 


16 


16 


14 


18 


15 


1 


1 


Macao-China 


17 


18 


14 


12 


19 


11 


6 


7 


Austria 


18 


19 


20 


18 


22 


22 


-4 


-3 


Belgium 


19 


17 


16 


14 


18 


14 


5 


3 


Ireland 


20 


20 


20 


18 


22 


18 


2 


2 


Flungary 


21 


22 


22 


20 


24 


21 


0 


1 


Sweden 


22 


21 


21 


19 


22 


21 


1 


0 


Poland 


23 


23 


23 


22 


25 


24 


-1 


-1 


Denmark 


24 


24 


25 


23 


27 


24 


0 


0 


France 


25 


25 


24 


20 


27 


25 


0 


0 


Croatia 


26 


27 


27 


25 


30 


28 


-2 


-1 


Iceland 


27 


26 


28 


25 


31 


29 


-2 


-3 


Latvia 


28 


29 


32 


29 


35 








United States 


29 


32 


33 


30 


35 








Slovak Republic 


30 


31 


34 


30 


36 


35 


-5 


-4 


Spain 


31 


28 


28 


25 


32 


29 


2 


-1 


Lithuania 


32 


34 


30 


25 


34 


28 


4 


6 


Norway 


33 


30 


29 


26 


32 


28 


5 


2 


Luxembourg 


34 


35 


34 


31 


35 


35 


-1 


0 


Russian Federation 


35 


33 


31 


26 


35 








Italy 


36 


36 


37 


35 


38 


37 


-1 


-1 


Portugal 


37 


37 


38 


37 


39 


37 


0 


0 


Greece 


38 


38 


36 


31 


38 


33 


5 


5 


Israel 


39 


39 


39 


38 


39 


39 


0 


0 


Chile 


40 


40 


41 


40 


42 


41 


-1 


-1 


Serbia 


41 


43 


43 


43 


44 


43 


-2 


0 


Bulgaria 


42 


42 


41 


40 


42 








Uruguay 


43 


41 


41 


40 


42 


41 


2 


0 


Turkey 


44 


45 


45 


44 


47 


45 


-1 


0 


Jordan 


45 


44 


44 


43 


45 


44 


1 


0 


Thailand 


46 


46 


47 


46 


49 








Romania 


47 


48 


46 


45 


48 


46 


1 


2 


Montenegro 


48 


50 


50 


48 


51 


49 


-1 


1 


Mexico 


49 


47 


47 


45 


49 


48 


1 


-1 


Indonesia 


50 


53 


53 


52 


54 


52 


-2 


1 


Argentina 


51 


49 


50 


49 


51 








Brazil 


52 


52 


52 


52 


54 








Colombia 


53 


51 


49 


47 


51 


49 


4 


2 


Tunisia 


54 


54 


53 


52 


54 


54 


0 


0 


Azerbaijan 


55 


55 


55 


55 


55 








Qatar 


56 


56 


56 


56 


56 


56 


0 


0 


Kyrgyzstan 


57 


57 


57 


57 


57 
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Table A5. Rankings based on new science items introduced in PISA 2006 



Country 


Initial 

report 

science 

literacy 

rank 


Percent 
correct ranking 
based on new 
PISA 2006 
items 


Rankings based solely on countries' 
preferred items 

Median Ran 9 e: Ran 9 e: 

rank lowe / u W e J 

rank rank 


Rank on own 
preferred new 
PISA 2006 
items 


Difference 
between initial 
report rank 
and preferred 
items rank 


Difference between 
percent-correct rank 
based on all items 
and preferred items 
rank 


Finland 


1 


1 


1 


1 


1 


1 


0 


0 


Flong Kong-China 


2 


2 


3 


2 


11 


2 


0 


0 


Canada 


3 


3 


3 


2 


5 


2 


1 


1 


Chinese Taipei 


4 


6 


8 


2 


25 


8 


-4 


-2 


Estonia 


5 


4 


6 


3 


14 


4 


1 


0 


Japan 


6 


5 


8 


3 


22 


7 


-1 


-2 


New Zealand 


7 


7 


5 


2 


12 


7 


0 


0 


Australia 


8 


8 


7 


4 


15 


6 


2 


2 


Netherlands 


9 


9 


7 


4 


16 


9 


0 


0 


Liechtenstein 


10 


10 


11 


7 


18 








Korea 


11 


11 


15 


8 


26 


13 


-2 


-2 


Slovenia 


12 


13 


11 


7 


20 


11 


1 


2 


Germany 


13 


12 


12 


9 


19 


11 


2 


1 


United Kingdom 


14 


14 


13 


8 


18 


13 


1 


1 


Czech Republic 


15 


15 


13 


5 


18 


13 


2 


2 


Switzerland 


16 


16 


18 


14 


23 








Macao-China 


17 


18 


20 


11 


23 


12 


5 


6 


Austria 


18 


19 


16 


9 


21 


17 


1 


2 


Belgium 


19 


17 


17 


12 


23 


13 


6 


4 


Ireland 


20 


20 


20 


13 


25 


13 


7 


7 


Flungary 


21 


22 


21 


14 


30 


20 


1 


2 


Sweden 


22 


21 


20 


12 


24 


30 


-8 


-9 


Poland 


23 


23 


23 


18 


28 


27 


-4 


-4 


Denmark 


24 


24 


26 


20 


33 


23 


1 


1 


France 


25 


25 


27 


20 


36 


33 


-8 


-8 


Croatia 


26 


27 


26 


15 


34 


29 


-3 


-2 


Iceland 


27 


26 


27 


21 


33 


28 


-1 


-2 


Latvia 


28 


29 


30 


24 


37 








United States 


29 


32 


30 


19 


35 








Slovak Republic 


30 


31 


28 


23 


33 


34 


-4 


-3 


Spain 


31 


28 


31 


24 


35 








Lithuania 


32 


34 


35 


25 


37 


31 


1 


3 


Norway 


33 


30 


31 


22 


35 


27 


6 


3 


Luxembourg 


34 


35 


33 


25 


36 


34 


0 


1 


Russian Federation 


35 


33 


31 


22 


35 








Italy 


36 


36 


36 


29 


38 


34 


2 


2 


Portugal 


37 


37 


36 


28 


38 


35 


2 


2 


Greece 


38 


38 


38 


34 


39 


37 


1 


1 


Israel 


39 


39 


39 


37 


40 


39 


0 


0 


Chile 


40 


40 


40 


39 


44 


41 


-1 


-1 


Serbia 


41 


43 


41 


39 


44 


41 


0 


2 


Bulgaria 


42 


42 


43 


41 


43 








Uruguay 


43 


41 


41 


40 


45 


41 


2 


0 


Turkey 


44 


45 


45 


43 


48 


44 


0 


1 


Jordan 


45 


44 


44 


43 


48 


44 


1 


0 


Thailand 


46 


46 


46 


42 


49 








Romania 


47 


48 


48 


45 


50 


50 


-3 


-2 


Montenegro 


48 


50 


49 


46 


51 


48 


0 


2 


Mexico 


49 


47 


47 


45 


52 


49 


0 


-2 


Indonesia 


50 


53 


53 


50 


54 


52 


-2 


1 


Argentina 


51 


49 


49 


48 


51 








Brazil 


52 


52 


52 


51 


54 








Colombia 


53 


51 


51 


47 


53 


50 


3 


1 


Tunisia 


54 


54 


54 


52 


55 


54 


0 


0 


Azerbaijan 


55 


55 


55 


45 


55 








Qatar 


56 


56 


56 


56 


57 


56 


0 


0 


Kyrgyzstan 


57 


57 


57 


56 


57 
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